The software testing phase is a critical phase in the software development process. During the software development process, the software testing phase occurs after the software has been designed, implemented in a programming language, and tested to a limited degree. During the testing phase, software testers test the software extensively to ensure that the software meets all of the requirements it is intended to meet. In order to accommodate simultaneous testing of several different software packages by several testers, multiple test machines are often implemented. Different types of software packages may need to be tested on different types of test machines, such as, for example, test machines with different hardware configurations and/or different operating systems. When a large number of software testers are required to share common resources for software testing, provisions must be made for scheduling the tests in order to efficiently manage these shared resources. The efficient management of these shared resources may also require that tests and the results of the tests be recorded so that the tests can be used repeatedly if needed and so that the results of the tests can be analyzed and subsequently used for comparison with the results of tests performed at a later time.
In an effort to maximize efficiency in the handling of test scheduling and test execution, attempts have been made to automate software testing by using a server to manage test machines and to allocate test packages among the test machines in accordance with a schedule. Sun Microsystems, Inc. has proposed an automated task-based scheduler for use with UNIX platform systems which uses scripts to allow users operating "client" machines to schedule tests to be executed on "target" machines. A central server receives a request from a client machine to perform a task. The server maintains information relating to all currently scheduled tasks on all target machines in a "status" database. The server maintains information relating to the expected duration of each test package and other test package attributes in a "packages" database. When the server receives a request to perform a task from a client machine, the server determines the loads on each of the target machines based on the expected duration of each test package and then schedules the task on the target machine with the least current load. A task file created at the client machine and copied to the server includes priority information relating to the task requested by the client machine. The target machine selects a task to be performed based on this priority information. Once a task is completed, the results are copied back to the server which compares them to a set of "golden results" and creates a comparison report which is mailed back to the user that requested the test.
One disadvantage of the system proposed by Sun Microsystems is that no provision is made for expanding the system outside of the LAN. Components to be added to the system cannot simply be placed on line outside of the LAN because no provision is made for communicating the scripts over other networks outside of the LAN. Also, if a test machine were added, no provision is made for automatically rescheduling tasks to allow the benefits of the new test machine to be immediately realized by the system, which reduces the flexibility and adaptability of the system. Rather, if a test machine is added to the system, the "machines" database will need to be manually updated to include information relating to the new test machine, such as its operating system and identity, so that the server can schedule tests on the new test machine.
Accordingly, a need exists for an automated testing system which is automatically expandable to allow system components to be easily and automatically added to the system, and which is distributed such that the users and components of the test system may be distributed over any network, such as the Internet.